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Summary 

This Report describes subjective tests carried out as part of the Eureka 95 project to 
compare different multi-channel sound systems being considered for use with HDTV. In the tests, 
the number of sound channels varied from one (mono) to five (three front channels and two 
surround channels). 

The improvement in quality obtained by increasing the number of front channels from two 
to three (adding a centre-front channel) was often similar to that obtained by increasing the number 
of front channels from one to two. Two-channel stereo was in one case found inferior to mono, 
presumably because of mislocation of central sound images for off-centre listeners, but three-channel 
stereo was always preferred to two-channel stereo (and to mono). 

The improvement obtained by adding one or two surround channels was about half that 
obtained by adding a centre-front channel On most programme items the use of two surround 
channels, rather than one, gave a small improvement but only on one programme item was this 
significant. 
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MULTI-CHANNEL SOUND FOR HDTV: 
SUBJECTIVE TESTS ON SOUND CHANNEL CONFIGURATION 

J.A. Fletcher, M.A. (Cantab.) 



1. INTRODUCTION 

It has been suggested that High Definition 
Television should include a sound system which 
provides a better listening environment than the two- 
channel stereophony used with present-day television. 
The ITU-R, in a Recommendation 1 , proposes a five- 
channel system, in which three channels are used for 
front loudspeakers positioned left, centre and right, and 
two channels are used for left and right surround loud- 
speakers. This system is denoted 3/2 because there are 
three front channels and two surround channels. 

The recommended loudspeaker arrangement is 
shown in Fig. 1. The surround loudspeakers are placed 
to subtend an angle of 120° - 160° behind the 
listener. In informal listening tests at BBC Research 
Department, this angle was found to be optimum 
when compared with 60° (a 'stereo pair' at the back) 
and 180° (loudspeakers at the sides of the listener). 

In the 3/2 system, the centre-front channel 
gives improved stabilisation of central sound images, 
particularly for off-centre listening positions, and the 
surround channels allow the portrayal of a more 
realistic acoustic atmosphere. 



were conducted jointly by BBC, Philips and Deutsche 
Bundespost Telekom as part of the Eureka 95 project. 
The purpose was to quantify the improvement gained 
by using a 3/2 system and to compare it with systems 
using fewer sound channels. Although the 3/2 channel 
configuration is widely accepted as the preferred 
multi-channel format, there had been little experi- 
mental work, other than scientific image location 
tests 2 ' 3 , to justify this choice. 

Two sets of subjective tests were carried out. 
The first took place in February 1992 and concerned 
both the centre-front channel and the addition of one 
or two surround channels. The second set of tests, in 
October 1992, was concerned only with frontal sound 
presentation. This time, the subjective benefit of 
increasing the number of front sound channels from 
two to three was compared with the benefit of 
increasing the number of channels from one to two 
(i.e. the well-known difference between mono and 
two-channel stereo). 

In the second set of tests a '2%' channel system 
which uses a half-bandwidth centre channel was also 
tested 4 . 



The subjective tests described in this Report 2. SUMMARY OF SYSTEMS UNDER TEST 

Multi-channel systems are denoted by two 
numbers separated by a slash (/); these being the 
number of front channels and the number of surround 
channels respectively. 

1/0 Mono replayed through a central loudspeaker. 

2/0 Two-channel stereo replayed through left and 
right loudspeakers. 

3/0 Three-channel stereo replayed through left, 
centre and right loudspeakers. 

2.5/0 Three-channel stereo replayed through left, 
centre and right loudspeakers, but with a 
centre channel of reduced bandwidth (8 kHz). 
The higher frequency signals from the original 
centre channel are fed to the outer 
loudspeakers. 

3/1 Three-channel stereo at the front with one 
surround channel replayed through both left 
and right surround loudspeakers. 




Fig. 1 - Reference loudspeaker arrangement with 
loudspeakers L/C/R andLS/RS. 
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3/2 Three-channel stereo at the front with two 
surround channels replayed through left and 
right surround loudspeakers. 



3. OBJECTIVES OF THE TESTS 

The first set of tests were divided into two 
parts. The first part used front loudspeakers only and 
was designed to measure the benefit of using a centre 
loudspeaker in addition to the conventional two- 
channel stereo pair. The second part was designed to 
measure the benefits of using one and two rear 
channels in addition to the front loudspeakers. 
The comparisons presented for each programme item 
were: 

2/0 vs. 3/0 
3/0 vs. 3/2 
3/1 vs. 3/2 

The second set of tests were designed to make 
fuller comparisons between the different channel 
configurations for frontal presentation. In particular, 
mono was included so that the well-known quality 
difference between mono and two-channel stereo 
could be used to judge the results of other 
comparisons. The 2.5/0 system was also included. 

The comparisons were chosen to compare each 
system with the next one in the hierarchy as well as to 
compare each system with a reference of 3/0. 

The comparisons presented for each programme 
item were: 

1/0 vs. 3/0 

2/0 vs. 3/0 

2.5/0 vs. 3/0 

1/0 vs. 2/0 

2/0 vs. 2.5/0 

4. TEST MATERIAL 

For these tests, it was decided that available 
real programme material would be used. This had the 
advantage that quality improvements observed during 
the tests would be representative of what a listener to 
an HDTV broadcast might experience. The other 
major advantage was that the tests could be prepared 
with the minimum of delay. The programme material 
available was limited and so did not necessarily 
contain items which fully exploited all aspects of the 
3/2 system. 

The tests used both sound and picture since in 



order to assess the accuracy of sound image location it 
is necessary to have a picture of the event itself. 

The source material was in 3/2 format. 
Versions of the programme items in formats with 
fewer channels were prepared by downmixing. 

The programme extracts used for the tests were 
of 15 s duration and are listed opposite. All the 
extracts were from BBC recordings except Zirkus 
Knie, which was kindly provided by the Institut fur 
Rundfunktechnik (IRT). 



5. TEST PROCEDURE 

The comparisons were made using an ABAB 
sequence of 15 s programme extracts. A and B were 
marked separately on a continuous quality-scale which 
was labelled, in analogy with the ITU-R 5-point 
quality scale*, from 'bad' to 'excellent'. The test 
instructions are given in Appendix 1. 

At the BBC, the second set of tests included an 
element of training, as now recommended by the 
ITU-R (for tests on small impairments) 5 . The 
participants were told that there were to be four 
different systems under test. A programme extract was 
then played through each system. This made the 
listeners aware of the range of differences they could 
expect to hear in the test. It also made it clear that, 
although many test comparisons would show clearly 
audible differences, it was important to listen for very 
subtle differences between test items (i.e. when 2.5/0 
and 3/0 were being compared). 

Both sets of tests were carried out by three 
organisations: BBC Research Department, Kingswood 
Warren; Philips Research Laboratories, Eindhoven; 
and Deutsche Bundespost Telekom, Berlin. 

The seating and loudspeaker layout used at 
BBC Research Department for the first set of tests is 
shown in Fig. 2 (see page 4). For the second set of 
tests, all three organisations used a common seating 
and loudspeaker layout, as shown in Fig. 3. 



6. RESULTS ANALYSIS 

From the original data, the differences between 
the judgements of versions A and B were calculated 
per subject and per item. For this calculation, the scale 
was taken to run linearly from (minimum quality) 
to 100 points (maximum quality). The maximal 
possible quality difference was therefore +100 points. 

See Appendix 1. Fig. A1.1. 
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The programme extracts used for the tests. 



Beethoven Violin Concerto 



Concert featuring violin soloist centrally positioned. 



Edinburgh Tattoo* 



Marching military bagpipe bands. The mix was 
limited by unsuitable microphone positions with the 
result that there is a similar sound from all 
directions. Occasional audience applause in 
surround channels but at low level relative to the 
music. 



FA Cup Football* 



Sequence includes a goal. Commentary midway 
between left and centre. Crowd noise in surround 
channels. 



The House of Eliott 



Drama. Conversation among three people seated at 
a restaurant table. Voices panned to correspond 
with seating positions, i.e. left of centre, centre, right 
of centre. Music and restaurant effects around. 



Tipsy Gipsy 



Humorous section from a 'Viennese Evening' 
Promenade Concert. Female vocalist centre front, 
some audience laughter. 



Vilia 



Section from 'Viennese Evening' Promenade 
Concert. Female vocalist centre front. 



Wimbledon Tennis 



Commentary midway between left and centre. 
Umpire midway between centre and right. Court 
effects centre front (not very well localised). 
Audience noise around. 



Zirkus Knie: acrobats 



Sequence involving an acrobat jumping onto an 
elephant. Includes the ringmaster's voice, drum roll 
from band, audience noise. 



Zirkus Knie: hammering peg 



Preparation for erection of a circus tent. A man is 
seen hammering a peg into the ground. Hammering 
and vehicle noise centre front, birdsong at sides. 



Wind in the Willows** 



Programme featuring the wildlife near a small 
stream. Noise of animal footsteps, sniffing, etc. 
centre front. Woodland atmosphere around. 



These items were not used in the second test because they were found not to demonstrate the features under test. 
** This item was not available at the time of the first test. 
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The mean quality difference among various groups of 
listeners was evaluated for each comparison. 

The standard deviation and standard error of 
the mean were also calculated. The standard deviation 
indicates the spread of responses from the listeners. 
The standard error of the mean indicates the certainty 
which can be attached to the measured mean value. 
For example, if the true mean preference of a large 
number of listeners is p., and the measured mean prefer- 
ence is fi m with standard error or mean x, then one 
can say with 95% confidence that /x>/i m — 1.65 X x. 



7. RESULTS FROM THE FIRST SET OF 
TESTS 

7.1 Frontal test: 3/0 vs. 2/0 

7.1.1 Overall 

Fig. 4 (overleaf) shows the mean and standard 
deviation of the results from all listeners. The mean 
quality difference is indicated by a bold line and the 
standard deviation is indicated by a shaded region 
extending + one standard deviation about the mean. 
Also shown are the percentages of listeners recording 
negative, zero (within 2 points) and positive 
preferences. 

Fig. 5 shows the same results with a shaded 
region extending + the standard error of the mean. 
This shows that the bounds of error on the calculated 
mean values are quite small. 

On all items there was a mean preference for 
the 3/0 presentation which varied according to the 
programme material from 6 to 24 points. The item 
Tipsy Gipsy showed the greatest mean preference. 

The smallest preference was recorded for the 
Edinburgh Tattoo; this is probably because there are 
no specific images in the frontal sound. Small amounts 
of preference were also recorded for the Tennis and 
the Football. These have clear images for comment- 
ators and umpire, which are left and right of centre 
respectively. However, because these image positions 
are somewhat arbitrary, the movements of these 
images which would be heard by off-centre listeners 
are not particularly detrimental. 

Large preferences were recorded on the other 
items; probably because they have clear central sound 
images (e.g. the solist in the Viennese Prom items and 
the centrally seated man in the House of Eliott 
sequence). 

The extent of the standard deviation shows 
that there was quite a wide spread of responses; 



nevertheless, the benefit of the 3/0 presentation is 
clear on the majority of items. For example, with 
Tipsy Gipsy 81% of listeners expressed a positive 
preference for 3/0. 

7.1.2 Variation with listener expertise 

The preference for 3/0 was greater among the 
expert listeners. 

7.1.3 Variation with seating position 

Figs. 6 and 7 show the results of this test 
separately for centre listeners and off-centre listeners. 
The preference for 3/0 was greater among the off- 
centre listeners, with the maximum mean preference 
increasing from 11 points to 27 points. This indicates 
that the addition of a centre-front channel does widen 
the listening area. 

7.2 Surround test: 3/2 vs. 3/0 

7.2.1 Overall 

Fig. 8 shows the mean and standard deviation 
of the results from all listeners. Fig. 9 shows the same 
results with a shaded region extending + the standard 
error of the mean. This shows that the bounds of error 
on the calculated mean values are quite small. 

On all items there was a mean preference for 
the 3/2 presentation which varied according to the 
programme material from 4 to 16 points. The item 
Zirkus Knie Acrobats showed the greatest mean 
preference and on this item 78% of listeners expressed 
a positive preference for 3/2. 

The smallest preferences were recorded with 
the Beethoven Violin Concerto and the Edinburgh 
Tattoo. In the former, the surround effect is subtle. In 
the latter, the surround channels are derived from 
unsuitably positioned microphones. 

Larger preferences were recorded on the other 
items. These were mostly spectator events at which a 
listener might like to imagine being present, but a 
similar preference for 3/2 was recorded on the drama 
item, where the surround channels conveyed ambient 
sounds from the scene. 

Once again, the extent of the standard 
deviation shows a fairly wide spread of responses. 
Many listeners were unfamiliar with 'surround sound' 
and personal taste for this type of presentation varied 
greatly. A few listeners found the 'surround sound' to 
be an unwelcome distraction. 

The amount of preference was smaller when 
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Fig. 4 - Preference for 3/0 over 2/0: mean results for all listeners with standard deviation. 
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Fig. 5 - Preference for 3/0 over 2/0: mean results for all listeners with standard error of mean. 
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-30-25-20-15-10 -5 5 10 15 20 25 30 35 40 45 50 55 60 
quality difference: mean +/- std. dev. 

Fig. 6 - Preference for 3/0 over 2/0: mean results for listeners on centre line with standard deviation. 
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Fig. 7 - Preference for 3/0 over 2/0: mean results for listeners off centre line with standard deviation. 
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quality difference: mean +/- std. dev. 

Fig. 8 - Preference for 3/2 over 3/0: mean results for all listeners with standard deviation. 
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Fig. 9 - Preference for 3/2 over 2/0: mean results for listeners with standard error of mean. 
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quality difference: mean +/- std. dev. 

Fig. 10 - Preference for 3/2 over 3/0: mean results for front row listeners with standard deviation. 
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quality difference: mean +/- std. dev. 

Fig. 11 - Preference for 3/2 over 3/0: mean results for middle row listeners with standard deviation. 
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-30-25-20-15-10 -5 5 10 15 20 25 30 35 40 45 50 55 60 
quality difference: mean +/- std. dev. 

Fig. 12 - Preference for 3/2 over 3/0: mean results for back row listeners with standard deviation. 
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quality difference: mean +/- std. dev. 

Fig. 13 - Preference for 3/2 over 3/1: mean results for all listeners with standard deviation. 
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Fig. 14 - Preference for 3/2 over 3/1: mean results for all listeners with standard error of mean. 



compared with the spread of responses than in the 
frontal test. However, the benefit of the surround 
channels remains clear on most of the programme 
items. 

7.2.2 Variation with listener expertise 

Both expert and non-expert listeners gave 
similar responses. 

7.2.3 Variation with seating position 

Figs. 10, 11 and 12 show the results of this test 
separately for listeners seated at the front, middle and 
back of the listening area relative to the 'reference 
position'. The preference for surround sound was 
greatest in the middle row of the listening area. This is 
the position in which the front-back balance is 
optimum and so can be considered most representative 



of a domestic listening environment, where the listener 
would optimise the front-back balance for the usual 
seating position. 

7.3 Surround test: 3/2 vs. 3/1 

7.3.1 Overai! 

Fig. 13 shows the mean and standard deviation 
of the results from all listeners. Fig. 14 shows the same 
results with a shaded region extending + the standard 
error of the mean. Although the preferences are small, 
most are statistically significant. 

There was a mean preference for 3/2 on all pro- 
gramme items although not significant on Beethoven 
Violin. The greatest mean preference was again 
recorded for the item Zirkus Knie Acrobats with 55% 
of listeners expressing a positive preference for 3/2. 
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The effect of different use of the surround 
channels among the different items was even more 
significant in this test. Many of the items were not 
good examples of the use of independent rear 
channels. For example, the subtle use of the surround 
channels in the violin concerto made differences 
between 3/2 and 3/1 very difficult to detect. More 
material would have been desirable for this 
comparison, such as an item with special effects used 
in the surround channels. 

Mean preferences for independent rear channels 
were small on many items for the reasons mentioned 
above. Many listeners could not hear any difference 
between the alternative presentations and in most cases 
the mean preference was small compared with the 
standard deviation. 

However, on the Zirkus Knie item, a preference 
of 10 points of the marking scale was observed. This 
item has a wide mix of audience applause on the 
surround channels with little correlation between left- 
rear and right-rear signals. This gave a realistic 
delocalisation of the sound which was lost when the 
rear channels were mixed to mono. 

7.3.2 Variation with listener expertise 

Expert listeners showed a greater preference for 
the 3/2 presentation. 

7.3.3 Variation with seating position 

The preference for independent surround 
channels was slightly greater for listeners towards the 
back of the listening area. 



8. RESULTS FROM THE SECOND SET OF 
TESTS 

8.1 2/0 vs. 1/0 

8.1.1 Overall 

Fig. 15 shows the mean and standard deviation 
of the results from all listeners. Fig. 16 shows the same 
results with standard error of the mean. Most of the 
measured preferences are 'significant'. That is, there is 
less than a 5% chance that the true preference averaged 
over a large number of listeners would be zero. 

Two-channel stereo was preferred to mono by 
approximately 15 to 20 points. An exception is the 
'House of Eliott' for which listeners preferred mono to 
two-channel stereo. This is probably because the mis- 
placement of the sound images of the actors which 
would have occurred for off-centre listeners in two- 
channel stereo was particularly objectionable. 



Other than House of Eliott, the lower 
preferences were for Beethoven, Vilia and Zirkus Knie 
Peg. All of these have the dominant sound centrally 
positioned so were better suited to mono presentation 
than some of the other items. 



8.1.2 Variation with listener expertise 

In the two-channel vs. mono comparison there 
was little difference between the results from expert 
and non-expert listeners. 



8.1.3 Variation with seating position 

Fig. 17 shows the results for all listeners sitting 
in the 'reference position' at the centre of the front 
row (seat 3). Fig. 18 shows the results from all 
listeners sitting adjacent to this seat in the front row 
(seats 2 and 4). The preference for two-channel stereo 
over mono was quite large in seat 3 but was 
considerably reduced in seats 2 and 4. There was quite 
a marked difference for such a small change in 
listening position. 

Even slightly off-centre there would have been 
a misplacement of images and conflict of visual and 
aural cues in the two-channel stereo presentation, 
which presumably detracted from the benefits of the 
wider sound stage. 

8.2 3/0 vs. 2/0 

8.2.1 Overall 

Fig. 19 shows the mean and standard deviation 
of the results from all listeners. Fig. 20 shows the same 
results with standard error of the mean. Most of the 
measured preferences are 'significant'. 

On all items there was a preference for 3/0 
over 2/0 but it varied from 3 to 28 points. For the 
majority of the items, however, it was in the range of 
approximately 10 to 20 points. 

The highest preference was recorded on the 
item House of Eliott which, as discussed earlier, suffers 
from mislocation of the central character's voice for off- 
centre listeners in the two-channel presentation. The 
lowest preference was recorded for Wimbledon which, 
as noted in the first set of tests, has dominant sound 
images off-centre (the commentator and umpire). 

8.2.2 Variation with listener expertise 

In the three-channel vs. two-channel com- 
parison the experts had a greater preference for three- 
channel (typically by about 10 points) than the 
non-experts. 
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Fig. 15 - Preference for 2/0 over 1/0: mean results for all listeners with standard deviation. 
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Fig. 16 - Preference for 2/0 over 1/0: mean results for all listeners with standard error of mean. 
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Fig. 17 - Preference for 2/0 over 1/0: mean results for listeners in seat 3, centre front with standard deviation. 
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Fig. 18 - Preference for 2/0 over 1/0: mean results for listeners in seats 2 and 4, next to centre front with standard deviation. 
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Fig. 19 - Preference for 3/0 over 2/0: mean results for all listeners with standard deviation. 
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Fig. 20 - Preference for 3/0 over 2/0: mean results for all listeners with standard error of mean. 
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Fig. 21 - Preference for 3/0 over 2/0: mean results for listeners in seat 3, centre front with standard deviation. 
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Fig. 22 - Preference for 3/0 over 2/0: mean results for listeners in seats 2 and 4, next to centre front with standard deviation. 



-ve 


zero 


+ve 


12 


27 


62 


8 


12 


81 


12 


12 


77 


15 


15 


69 


23 


38 


38 


8 


27 


65 


27 


15 


58 


15 


12 


73 



8.2.3 Variation with seating position 

Figs. 21 and 22 show the results for on- and 
off-centre listeners respectively. In the central position, 
seat 3, the preference for three-channel stereo over 
two-channel was quite small. This is to be expected 
because in a central listening position there should be 
good sound imaging even in the two-channel presenta- 
tion; the only difference being that with three channels 
central images will be 'real' rather than 'phantom'. 

A much greater preference was recorded off 
centre, in seats 2 and 4, where there should have been 
much better location of sound images in the three- 
channel presentation. 



8.3 Other comparisons 

The inclusion of a 1/0 vs. 3/0 comparison in 
the test allowed the additive nature of the results to be 



investigated. The difference between 3/0 and 1/0 was 
found to be approximately equal to the difference 
between 3/0 and 2/0 plus the difference between 2/0 
and 1/0. This is shown in Fig. 23 (overleaf); Zirkus 
Knie Peg is one exception. 



8.4 3/0 vs. 2.5/0 

Fig. 24 (page 15) shows the mean and 
standard deviation of the results from all listeners for 
the 3/0 vs. 2.5/0 comparison. The distribution of 
results was approximately centred on zero. 



8.5 Comparison of results from the 
different organisations 

The results from the three organisations were 
compared to see if their results differed significantly. 
This comparison is detailed in Appendix 2. There 
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Fig. 23 - Addition of results. 
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Fig. 24 - Preference for 3/0 over 2.5/0: mean results for all listeners with standard deviation. 



were some apparently large differences but in most 
cases the differences were not 'significant' at the 5% 
level. It does appear, however, that Philips' listeners 
consistently showed a smaller preference for 2/0 over 
1/0 than the other two groups. 



9. COMPARISON OF THE RESULTS FROM 
THE FIRST AND SECOND SETS OF 
TESTS 

Both sets of tests included a comparison of 3/0 
and 2/0, using many of the same programme items. 
The instructions for both tests were very similar and 
both used the same marking scale. It is interesting to 
compare the results. 

The preference for 3/0 over 2/0 was greater 
in the first test than in the second test. For example, 
with the items Vilia and Tipsy Gipsy the preference 
for 3/0 over 2/0 was about 23 points in the first test 
and about 16 points in the second test. 

The most likely reason for the different results 
is that the first test included only two- and three- 
channel front presentation, whereas the second included 
systems ranging from one to three channels. The wider 
range of systems in the second test probably resulted 
in a compression of the marking scale. 



10. CONCLUSIONS 

In the first set of subjective tests it was found 
that the improvement obtained by adding a centre 
loudspeaker to a two-channel stereo pair was approx- 
imately one grade of the 5-point quality scale on 
programme material with central sound images. The 
addition of two surround channels, to a frontal-only 



presentation, produced an improvement of approx- 
imately half a grade of the 5-point quality scale on 
programme material with specific ambient sounds in 
the surround channels. 

The use of two surround channels is intended 
to allow a spread of surrounding sound images to be 
reproduced in a way which cannot be achieved with a 
mono surround channel. In the subjective test this bene- 
fit was not demonstrated very clearly. It was felt that 
the programme material used was not the most suitable 
for this comparison. However, on one item containing 
audience applause an improvement of about one-third 
of a grade of the 5-point quality scale was recorded. 

The second set of tests (frontal-only) showed 
that, in general, the improvement in quality obtained 
by increasing the number of front channels from 2 to 
3 is similar to the improvement obtained by increasing 
the number of front channels from 1 to 2. For 
listeners who were off centre (even by only one 
seating position), the improvement obtained by increas- 
ing the number of channels from 2 to 3 is greater (on 
6 out of 8 programme items) than that obtained by 
increasing the number of channels from 1 to 2. 

Two-channel stereo may in some cases be 
inferior to mono; presumably this is because of the 
mislocation of central sound images for off-centre 
listeners. Three-channel stereo is always an improve- 
ment on two-channel stereo (and mono) and is most 
beneficial on items with central sound images. 

With the test material used, the average results 
showed no significant difference between 2.5/0 and 
3/0*. 



This leaves open the possibility that other test material and special 
test methods related to small impairments might show a significant 
difference. 
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APPENDIX 1 
Test Instructions 



Firs! set of tests: frontal test 



In this subjective test you are asked to assess different Sound Systems for High Definition Television. This test 
concentrates on the use of front loudspeakers. 

For each programme item presented, there will be two versions A and B each lasting 15 seconds. These will be 
shown in the sequence A,B,A,B. After this there will be a pause of 10 seconds for marking. You should mark both 
A and B on the five point quality scale. Do this with short horizontal lines on the results sheet as shown in the 
example. 

The session begins with an example illustrating the test procedure and the differences which may be evident. 

You should consider the locations and distribution of sound sources and the definition of these frontal sound 
images (e.g. clarity, sharpness) compared with those at the performance itself. 

You should not consider any technical impairments of the sound and picture quality. 

First set of tests: surround test 

In this subjective test you are asked to assess different Sound Systems for High Definition Television. This test 
concentrates on the use of additional loudspeakers towards the rear carrying surround information in different 
ways. 

For each programme item presented, there will be two versions A and B each lasting 15 seconds. These will be 
shown in the sequence A,B,A,B. After this there will be a pause of 10 seconds for marking. You should mark both 
A and B on the five point quality scale. Do this with short horizontal lines on the results sheet as shown on the 
results sheet. 

The session begins with a one minute example of Surround Sound for High Definition Television, followed by one 
example illustrating the test procedure and the differences which may be evident. 

You should assess the sensation of reality considering the distribution of ambient sounds around you compared 
with those at the performance itself. 

You should not consider any technical impairments of the sound and picture quality. 

Second set of tests (frontal presentation) 

In this subjective test you are asked to assess different Sound Systems for High Definition Television. This test 
concentrates on the use of front loudspeakers. 

For each programme item presented, there will be two versions A and B each lasting 15 s. These will be shown in 
the sequence A,B,A,B. After this there will be a 10 s pause for marking. You should mark both A and B on the 
five point quality scale. Do this with short horizontal lines on the results sheet as shown in the example. 

You should consider the locations and distribution of sound sources compared with those at the event 
itself, and the definition of these frontal sound images. 

The test begins with an example illustrating the test procedure and the differences which may be evident. 

There are two sessions, each lasting about 28 minutes. You should sit in the same place for both sessions. 
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APPENDIX 2 
Comparison of Results from Different Organisations 

Suppose two groups, A and B, do an identical test with mean results /xa and jub, and standard errors of 
mean xa and xb. If the two groups and test conditions are equivalent, then the expected value of the difference 
between the mean results of the two groups is zero. The expected standard deviation of this difference is 

(xi + xi) 1/2 . 

One can compare the actual difference with this expected distribution to see how likely it is that the 
difference arose by chance between equivalent groups. For example, if the mean results of the two groups differ by 
1.96 X the standard deviation above, then there is only a 5% probability that the result would have arisen by 
chance with equivalent groups. If the probability is less than 5%, it suggests that the groups may not in fact have 
been equivalent. 

This calculation is shown below comparing the results of the Philips and Telekom groups for the 
2/0 vs. 1/0 comparisons. 



Beethoven 


Philips 


Telekom 




mean 


std error 


mean 


std error 


difference 


std dev 


prob (%) 


7.0 


5.3 


11.5 


4.1 


-4.5 


6.7 


50.2 


House of Eliott 


-20.6 


5.3 


-6.1 


5.5 


-14.5 


7.6 


5.8 


Tipsy Gipsy 


7.2 


5.9 


21.2 


3.9 


-14.0 


7.1 


4.8 


Vilia 


3.8 


4.3 


10.5 


4.2 


-6.7 


6.0 


26.5 


Wimbledon 


9.9 


6.9 


25.4 


4.0 


-15.5 


8.0 


5.2 


Wind in the Willows 


11.9 


4.4 


28.2 


3.4 


-16.3 


5.6 


0.3 


Zirkus: Acrobats 


15.5 


5.5 


25.9 


4.2 


-10.4 


6.9 


13.3 


Zirkus: Peg 


5.2 


5.1 


15.0 


5.2 


-9.8 


7.3 


17.8 



A greater degree of consistency is shown between the Telekom and BBC groups: 



Beethoven 


Telekom 


BBC 




mean 


std error 


mean 


std error 


difference 


std dev 


prob (%) 


11.5 


4.1 


16.8 


3.7 


-5.3 


5.5 


33.7 


House of Eliott 


-6.1 


5.5 


-3.2 


4.8 


-2.9 


7.3 


69.1 


Tipsy Gipsy 


21.2 


3.9 


23.5 


3.7 


-2.3 


5.4 


66.9 


Vilia 


10.5 


4.2 


14.6 


3.9 


-4.1 


5.7 


47.4 


Wimbledon 


25.4 


4.0 


22.2 


3.8 


3.2 


5.5 


56.2 


Wind in the Willows 


28.2 


3.4 


15.7 


3.7 


12.5 


5.0 


1.3 


Zirkus: Acrobats 


25.9 


4.2 


25.7 


3.3 


0.2 


5.3 


97.0 


Zirkus: Peg 


15.0 


5.2 


19.1 


3.3 


-4.1 


6.2 


50.6 
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